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CLAIMS 

1. Apparatus for associating gaze data representative 
of the location of gaze of an operator with 
classifications of images of an operator looking at 
different locations, comprising: 

a receiver operable to receive a video stream 
defining a sequence of images representative of an 
operator at different points in time; 

a classification unit operable to assign one of a 
number of classifications to images in the video stream 
received by said receiver, said classification unit being 
operable to assign the same classifications to images of 
an operator looking at the same locations; and 

a calibration unit comprising: 

a classification store configured to store data 
identifying the pairs of different classifications 
assigned to pairs of images in a video stream received 
--by said receiver representative of an operator at 
different times separated by less than a preset time 
period; 

a gaze conversion store configured to store data 
associating each of said number of classifications with 
gaze data representative of a gaze location; and 
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an update unit operable to update gaze data stored 
in said gaze conversion store by updating said gaze data 
such that gaze data for pairs of different 
classifications identified by data stored in said 
classification store identify gaze locations which are 
closer together and gaze data for pairs of different 
classifications not identified by data stored in said 
classification store identify gaze locations which are 
further apart. 

2. Apparatus in accordance with claim 1 wherein said 
calibration unit is responsive to the assignment of 
different classifications to images representative of an 
operator at different times separated by less than said 
pre set time period by said classification unit to 
determine whether data stored in said classification 
store is indicative of said pair of classifications and 
to update data stored in said classification store in 
dependence upon said determination. 

3. Apparatus in accordance with claim 2 wherein said 
classification unit is operable to determine whether 
classifications assigned to consecutive images in a video 
stream received by said receiver are assigned to 
different classifications and to update data identifying 
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pairs of classifications stored in said classification 
store if the different classifications for a pair of 
consecutive images are not identified by data in said 
classification store . 

5 

4 . Apparatus in accordance with any claim 1 wherein said 
update unit is operable to determine the distance between 
locations identified by gaze data associated with pairs 
of classifications, said update unit being operable to 
10 update gaze data on the basis of said determined 

distances and said data stored in said classification 
store . 



5. Apparatus in accordance with claim 4 wherein said 
15 update unit is operable to adjust gaze data associated 

with classifications identified by data in said 
classification store when the determined distance between 
the gaze locations identified by gaze data for said pair 
of classifications is greater than a predetermined 
20 threshold • 



6 . Apparatus in accordance with claim 4 wherein said 
update unit is operable to adjust gaze data associated 
with classifications not identified by data in said 
2 5 classification store when the determined distance between 
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the gaze locations identified by gaze data for said pair 
of classifications is less than a predetermined 
threshold. 

7. Apparatus in accordance with claim 1 wherein said 
gaze data comprises a co-ordinate data, said update unit 
being operable to update gaze data for classifications 
by adjusting said co-ordinate data by an amount 
determined utilising the difference between the co- 
ordinates associated with selected pairs of 
classifications • 

8. Apparatus in accordance with claim 7 wherein said 
update unit us operable to adjust gaze data associated 
with a pair of classifications identified by data in said 
classification store when the distance between the 
locations identified by said gaze data is more than a 
predetermined threshold by adjusting the gaze data for 
at least one of said pair of classifications utilising 
the following equation: 



P update 




where 
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Pupdate 



is a vector representing the updated gaze data 



Pold 



is a vector representing the data being updated 



is a vector representing the gaze location data 



of the other point in the pair of classifications 
and o is a predetermined value. 

9. Apparatus in accordance with claim 7 wherein said 
update unit is operable to adjust gaze data associated 
with a pair of classifications not identified by data in 
said classification store, when the distance between the 
locations identified by said gaze data is less than a 
predetermined threshold by adjusting the gaze data for 
at least one of said pair of classifications utilising 
the following equation: 



update 




where 
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5 



— is a vector representing the updated gaze data 

P update 



— is a vector representing the data being updated 

Pold 



— is a vector representing the gaze location data 

P other 



of the other point in the pair of classifications 



and a is a predetermined value. 



10. Apparatus in accordance with claim 1 further 
comprising: 

10 a display operable to display a user interface 

including a pointer; 

a control device operable to control the location 
of said pointer on said display , said control device 
being operable to enable a user to select portions of 

15 said display utilising said pointer; and 

a monitoring unit operable to monitor the selection 
of portions of said display utilising said control 
device, and to update gaze data associated with the 
classification of an image an operator obtained when a 

20 selection of a portion of the display is made using the 
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control device utilising data identifying the location 
of the portion of the display selected. 



11. Apparatus in accordance with the claim 10 wherein 
5 said control device comprises a mouse, said selection of 

a portion of the display being initiated by clicking a 
button on said mouse. 



12. Apparatus in accordance with claim 1 further 
10 comprising: 

a display operable to display a plurality of user 
interfaces identifying particular portions of said 
display, 

wherein said calibration unit is operable to 
15 associate classifications of images obtained when a user 

interface identifying a particular portion of the display 
is shown with gaze data identifying the location of the 
portion of said display identified by said user 
interface. 

20 

13. Apparatus in accordance with claim 1 further 
comprising a data store operable to store data 
identifying the frequency with which images are 
associated with each of said classifications, said 

25 calibration unit being operable to update gaze data 
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stored in said second data store by applying a selected 
transformation to said gaze data on the basis of said 
data stored in said data store. 

14. Apparatus in accordance with claim 12 wherein said 
selected transformation comprises a transformation 
defining a ref lection , a rotation , a translation or a 
combination of a reflection , rotation and a translation. 

15. Apparatus in accordance with claim 1 wherein said 
classification unit comprises a self -organising map. 

16. Apparatus in accordance with claim 1 further 
comprising an output unit operable to output for a video 
stream received by said receiver a sequence of gaze data 
comprising gaze data associated by said gaze conversion 
store with the classifications assigned to each image in 
said video stream by said classification unit. 

17. A method associating gaze data representative of the 
location of gaze of an operator with classifications of 
images of an operator looking at different locations, 
comprising: 
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receiving a video stream defining a sequence of 
images representative of an operator at different points 
in time; 

assigning one of a number of classifications to 
images in the received video stream, wherein the same 
classifications are assigned to images of an operator 
looking at the same locations; 

storing data identifying the pairs of different 
classifications assigned to pairs of images in a received 
video stream representative of an operator at different 
times separated by less than a preset time period; 

storing data associating each of said number of 
classifications with gaze data representative of a gaze 
location; and 

updating stored gaze data by updating said gaze data 
such that gaze data for pairs of different 
classifications identified by stored data said gaze data 
identifies gaze locations which are closer together and 
gaze data for pairs of different classifications not 
identified by stored data are updated to identify gaze 
locations which are further apart. 

18. A method in accordance with claim 17 further 
comprising responding to the assignment of different 
classifications to images representative of an operator 



88 2792850(03) 

at different times separated by less than said pre set 
time period by determining whether stored data identifies 
said pair of classifications and updating said stored 
data in dependence upon said determination. 

19. A method in accordance with claim 18 wherein said 
determination comprises determining whether 
classifications assigned to consecutive images in a 
received video stream are assigned to different 
classifications and updating said stored data comprises 
storing data identifying the pair of different 
classifications if the different classifications for a 
pair of consecutive images are not already identified by 
stored data. 

20. A method in accordance with claim 17 wherein said 
updating stored gazed data comprises determining the 
distance between locations identified by gaze data 
associated with pairs of classifications, and updating 
gaze data on the basis of said determined distances and 
said stored data. 

21. A method in accordance with claim 20 wherein said 
updating stored gaze data comprises adjusting gaze data 
associated with classifications identified by stored data 
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when the determined distance between the gaze locations 
identified by gaze data for said pair of classifications 
is greater than a predetermined threshold. 

22. A method in accordance with claim 20 wherein said 
updating stored gaze data comprises adjusting gaze data 
associated with classifications not identified by stored 
data when the determined distance between the gaze 
locations identified by gaze data for said pair of 
classifications is less than a predetermined threshold. 

23. A method in accordance with claim 17 wherein said 
gaze data comprises a co-ordinate data and said updating 
stored gaze data comprises updating gaze data for 
classifications by adjusting said co-ordinate data by an 
amount determined utilising the difference between the 
co-ordinates associated with selected pairs of 
classifications . 

24. A method in accordance with claim 23 wherein said 
updating stored gaze data comprises adjusting gaze data 
associated with a pair of classifications identified by 
stored data when the distance between the locations 
identified by said gaze data is more than a predetermined 
threshold by adjusting the gaze data for at least one of 
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said pair of classifications utilising the following 
equation: 

Pupdate Pold CT ^ Pother Pold > 

where 

— is a vector representing the updated gaze data 

Pupdate 

— is a vector representing the data being updated 

Pold 

— is a vector representing the gaze location data 

Pother 

of the other point in the pair of classifications 
and a is a predetermined value. 

25. A method in accordance with claim 23 wherein said 
updating stored gaze data comprises adjusting gaze data 
associated with a pair of classifications not identified 
by stored data when the distance between the locations 
identified by said gaze data is less than a predetermined 
threshold by adjusting the gaze data for at least one of 
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said pair of classifications utilising the following 
equation: 

_ = _ + ±(_ __ ) 

Pupdate Pold & V Pother Pold / 

where 

— is a vector representing the updated gaze data 

Pupdate 

— is a vector representing the data being updated 

Pold 

— is a vector representing the gaze location data 

Pother 

of the other point in the pair of classifications 
and a is a predetermined value, 

26. A method in accordance with claim 17 further 
comprising the steps of: 

displaying a user interface including a pointer; 

a monitoring user input to identify the selection 
of portions of said user interface utilising said pointer 
under the control of a control device, and updating 
stored gaze data associated with the classification of 
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an image an operator obtained when a selection of a 
portion of the user interface is determined to have been 
made utilising data identifying the location of the 
portion of the user interface selected with said pointer. 

27. A method in accordance with the claim 26 wherein 
said control device comprises a mouse, said selection of 
a portion of the display being initiated by clicking a 
button on said mouse. 

28. A method in accordance with claim 17 further 
comprising the steps of: 

displaying a plurality of user interfaces 
identifying particular portions of a display; and 

associating classifications of images obtained when 
a user interface identifying a particular portion of the 
display is shown with gaze data identifying the location 
of the portion of said display identified by said user 
interface. 

29. A method in accordance with claim 17 further 
comprising the steps of: 

storing data identifying the frequency with which 
images are associated with each of said classifications; 
and 
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updating gaze data by applying a selected 
transformation to said gaze data on the basis of said 
data identifying the frequency with which images are 
associated with each of said classifications. 

5 

30. A method in accordance with claim 2 9 wherein said 
selected transformation comprises a transformation 
defining a reflection, a rotation, a translation or a 
combination of a reflection, rotation and a translation. 

10 

31. A method in accordance with claim 17 further 
comprising the steps of: 

outputting for a received video stream a sequence 
of gaze data comprising gaze data associated with the 
15 classifications assigned to each image in said video 

stream. 

32. A data carrier storing computer implementable 
process steps for causing a programmable computer to 

20 perform a method in accordance with claim 17. 



33. A data carrier in accordance with claim 32 
comprising a computer disc. 
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34. A data carrier in accordance with claim 32 
comprising an electric signal transferred via a network. 

35. A computer disc in accordance with claim 33 wherein 
said computer disc comprises an optical , magneto-optical 
or magnetic disc. 

36. Apparatus for classifying images comprising: 

a receiver operable to receive image data defining 
images; 

a patch generation module operable to generate for 
a number of points in an image defined by image data 
received by said receiver, image patches , said image 
patches being derived by processing image data defining 
portions of said image which include said points, a 
plurality of different image patches being derived for 
each of said points; 

a patch comparison module operable to compare stored 
data with image patches generated by said patch 
generation module to identify stored data which most 
closely corresponds to said generated patches; and 

a classification unit for generating a 
classification of an image defined by image data received 
by said receiver, wherein said classification comprises 
data identifying points in an image and for each of said 
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points , data identifying stored data determined to most 
closely correspond to an image patch derived by said 
patch generation module from image data defining a 
portion of the image including said point. 

37. A method of classifying images comprising: 

receiving image data defining images; 

generating for a number of points in an image 
defined by received image data, image patches , said image 
patches being derived by processing image data defining 
portions of said image which include said points , a 
plurality of different image patches being derived for 
each of said points; 

comparing stored data with said generated image 
patches to identify stored data which most closely 
corresponds to said generated patches; and 

generating a classification of an image defined by 
received image data, wherein said classification 
comprises data identifying points in an image and for 
each of said points, data identifying stored data 
determined to most closely correspond to an image patch 
generated from image data defining a portion of the image 
including said point. 
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38. Apparatus for comparing colour images independently 
of the illumination of a subject object in said. images, 
comprising: 

a receiver operable to receive image data defining 
images , said image data comprising colour data for a 
plurality of pixels representative of the colours of a 
subject object appearing in said images; 

a processing unit operable to derive from image data 
received by said receiver , a colour reflectance image 
said colour reflectance image comprising colour data for 
said plurality of pixels representative of the 
contribution to the colour of a subject object in an 
image not arising due to the colour of the illumination 
of the subject object in said image , said processing unit 
being operable to derive said colour reflectance image 
such that the ratios of colour data of pixels in said 
colour reflectance image for a said subject object are 
independent of the colour of the illumination of said 
subject object in said image; and 

a comparator operable to compare generated colour 
reflectance images with stored image data to determine 
the correspondence between said stored data and said 
generated images; 

wherein said correspondence is determined on the 
basis of the comparison of the ratios of colour data for 
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pixels in a said generated colour reflectance image and 
corresponding ratios of colour data for pixels of said 
stored image data. 

39. A method of comparing colour images independently 
of the illumination of a subject object in said images / 
comprising: 

receiving image data defining images, said image 
data comprising colour data for a plurality of pixels 
representative of the colours of a subject object 
appearing in said images; 

processing a received image to derive from received 
image data a colour reflectance image said colour 
reflectance image comprising colour data for said 
plurality of pixels representative of the contribution 
to the colour of a subject object in an image not arising 
due to the colour of the illumination of the subject 
object in said image , said processing being such to 
derive said colour reflectance image such that the ratios 
of colour data of pixels in said colour reflectance image 
for a said subject object are independent of the colour 
of the illumination of said subject object in said image; 
and 
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comparing generated colour reflectance images with 
stored image data to determine the correspondence between 
said stored data and said generated images; 

wherein said correspondence is determined on the 
basis of the comparison of the ratios of colour data for 
pixels in a said generated colour reflectance image and 
corresponding ratios of colour data for pixels of said 
stored image data. 

40. Apparatus for associating data indicative of the 
orientation of a person's head with classifications of 
images of a person's head comprising: 

a receiver operable to receive a video stream 
defining a sequence of images of a person's head; 

a classification unit operable to assign one of a 
number of classifications to images in the video stream 
received by said receiver, said classification unit being 
operable to assign the same classifications to images of 
a person's head in substantially the same orientations; 
and 

a calibration unit comprising: 

a classification store configured to store data 
identifying pairs of different classifications assigned 
to pairs of images in a video stream received by said 
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receiver representative of a person's head at different 
times separated by less than a reference time period; 

a conversion store configured to store data 
associating each of said number of classifications with 
data representative of at least one head orientation; and 

an update unit operable to update data stored in 
said conversion store by updating said data such that 
data for pairs of different classifications identified 
by data stored in said classification store identify head 
orientations which are closer together and data for pairs 
of different classifications not identified by data 
stored in said classification store identify head 
orientations which are further apart. 

41. A method of associating data indicative of the 
orientation of a person's head with classifications of 
images of a person's head comprising: 

receiving a video stream defining a sequence of 
images of a person's head; 

assigning one of a number of classifications to 
images in the received video stream, wherein the same 
classifications are assigned to images of a person's head 
in substantially similar orientations; 

storing pair data identifying pairs of different 
classifications assigned to pairs of images in a received 
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video stream representative of a person's head at 
different times separated by less than a reference time 
period; 

storing association data associating each of said 
number of classifications with data representative of at 
least one head orientation; and 

updating said association data such that association 
data for pairs of different classifications identified 
by stored pair data identify head orientations which are 
closer together and association data for pairs of 
different classifications not identified by stored pair 
data identify head orientations which are further apart. 

42. Apparatus for associating data indicative of the 
viewing direction of a person's eyes with classifications 
of images of a person's eyes comprising: 

a receiver operable to receive a video stream 
defining a sequence of images of a person's eyes; 

a classification unit operable to assign one of a 
number of classifications to images in the video stream 
received by said receiver, said classification unit being 
operable to assign the same classifications to images of 
a person's eyes in substantially the same viewing 
directions; and 

a calibration unit comprising : 
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a classification store configured to store data 
identifying pairs of different classifications assigned 
to pairs of images in a video stream received by said 
receiver representative of a person's eyes at different 
times separated by less than a reference time period; 

a conversion store configured to store data 
associating each of said number of classifications with 
data representative of at least one viewing direction; 
and 

an update unit operable to update data stored in 
said conversion store by updating said data such that 
data for pairs of different classifications identified 
by data stored in said classification store identify 
viewing directions which are closer together and data for 
pairs of different classifications not identified by data 
stored in said classification store identify viewing 
directions which are further apart. 

43- A method of associating data indicative of the 
viewing directions of a person's eyes with 
classifications of images of a person's eyes comprising: 

receiving a video stream defining a sequence of 
images of a person's eyes; 

assigning one of a number of classifications to 
images in the received video stream, wherein the same 
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classifications are assigned to images of a person's eyes 
in substantially similar viewing directions; 

storing pair data identifying pairs of different 
classifications assigned to pairs of images in a received 
video stream representative of a person's eyes at 
different times separated by less than a reference time 
period; 

storing association data associating each of said 
number of classifications with data representative of at 
least one viewing direction; and 

updating said association data such that association 
data for pairs of different classifications identified 
by stored pair data identify viewing directions which are 
closer together and association data for pairs of 
different classifications not identified by stored pair 
data identify viewing directions which are further apart. 



